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Tests have been conducted at Bell Laboratories within the last 10 years 
to obtain subjective evaluations of the effects of loss, noise, and talker echo 
on telephone transmission quality. We use these subjective test results to 
formulate graphical and analytical models of subjective opinion that can 
be used in network planning studies to evaluate transmission performance 
of the network and to study the effects of network changes on performance. 
These models are based on the concept of a generalized transmission- 
rating scale. Separate opinion curves for each test take into account differ- 
ences caused by factors such as subject group, type of test, and range of 
conditions. We also describe the methods of data analysis used in the 
formulation of the transmission-rating scale and opinion models, provide 
a comparison of the test results with the models, and discuss the models in 
sufficient detail to permit their application in transmission planning 
studies. 

I. INTRODUCTION 

In the last 10 years, several tests have been conducted at Bell 
Laboratories to determine the subjective evaluation of loss, noise, and 
talker echo in telephone connections. The purpose of these tests was 
to obtain information for use in network planning studies. Several 
hundred volunteers served as subjects and participated in several 
thousand test calls with various amounts of loss, noise, and talker 
echo. Some of these tests were conducted on normal business calls 
made by Bell Laboratories employees; others were conducted in a 
laboratory environment. At the end of each call, the subject was asked 
to indicate his or her opinion of the transmission quality on a five- 
category rating scale : excellent, good, fair, poor, and unsatisfactory. 

The results from these tests were used to formulate the graphical 
and analytical models of opinion which are presented in this paper. 
These models have been used extensively in network planning studies 
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to evaluate present network performance and to study the effect of 
possible changes in the network. Although many of these studies are 
not yet published, a recent paper by Spang 1 provides an excellent 
example of the application of these models in toll transmission planning. 

For many years the Bell System has used subjective tests to obtain 
information on the effects of transmission quality which could be used 
in network planning studies. For example, the present Via Net Loss 
design plan, which introduces loss to control talker echo in the Direct 
Distance Dialing (ddd) network, is based on the results of subjective 
tests for talker echo that are included in a 1953 paper by Huntley. 2 
Subsequent talker-echo tests were described by Phillips in 1954. 3 
Coolidge and Reier reported the results of tests of received telephone 
speech volume in 1959 and introduced the concept of volume grade-of- 
service. 4 Test results for message circuit noise were used in noise 
grade-of-service studies described by Lewinski in 1964. 5 

Most of the tests mentioned considered the effect of one transmission 
parameter at a time. Since transmission parameters appear in com- 
binations and there are, in many instances, important interactions, a 
new series of conversational tests for the combined effects of connection 
loss and circuit noise was initiated in 1965. Subjective evaluations were 
obtained on normal business calls within Bell Laboratories. The results 
of these tests were reported by Sen in 1971. 6 Since then, the use of 
combined loss-noise grade-of-service based on these tests has largely 
replaced the use of the earlier noise and volume grade-of-service. 

Another test to determine subjective reaction to loss and noise was 
conducted in 1969 on normal business calls within the Bell Laboratories 
location at Holmdel, New Jersey. There were two reasons for the new 
test. One was to obtain a larger number of subjects per test condition 
and thus reduce the experimental variability. The other reason was to 
include both symmetric loss conditions, as was done in the 1965 tests, 
and asymmetric loss conditions; i.e., unequal loss in the two directions 
of transmission. The test results for the symmetric conditions indicated 
a more critical assessment of qualitj' than the 1965 tests, which could 
not be explained by known differences in the tests. Therefore, a third 
test was planned and conducted in 1972. 

During this same period of time, new talker-echo tests were initiated. 
These echo tests used the same five-category rating scale as the loss- 
noise tests so that possible tradeoffs between loss-noise and echo grade- 
of-service could be studied. Some of the echo tests were conducted in 
the laboratory. Others were conducted on normal business conversa- 
tions between Bell Laboratories employees, where values of loss, noise, 
echo-path loss, and echo-path delay could all be controlled. 

As the results of the various tests became available, work was con- 
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tinued to modify and improve earlier loss-noise opinion models and to 
develop similar talker-echo opinion models. Systematic methods were 
formulated to analyze the results from individual tests and to combine 
the results from different tests into a composite loss-noise-echo opinion 
model. In addition, a transmission-rating scale was introduced that 
assigned a single numerical value to any specific combination of trans- 
mission conditions. 

The concept of a generalized transmission-rating scale recognized 
that subjective test results can be affected by various factors such as 
the subject group, the type of test, and the range of conditions that 
are included in the test. These factors were found to cause changes in 
both the mean opinion score for a given condition and in the standard 
deviation.* Thus, there were difficulties in trying to establish a unique 
relationship between a given transmission condition and subjective 
opinion in terms of mean opinion score or other subjective measures 
of transmission quality. The introduction of a transmission-rating 
scale tended to reduce this difficulty by separating the relationship 
between transmission characteristics and opinion ratings into two 
parts. For the first part, the transmission rating as a function of the 
transmission characteristic, was anchored for two specific transmission 
conditions and thus tended to be much less dependent on individual 
tests. The second part, the relationship between transmission rating 
and subjective opinion ratings, could then be displayed for the in- 
dividual test. 

The essential features of the transmission-rating scale and opinion 
models are summarized in Section II of this paper in enough detail 
to permit their application in transmission planning studies. The re- 
mainder of the paper describes three subjective tests for connection 
loudness loss and circuit noise and four tests for talker echo, outlines 
the methods of analysis, and describes the formulation of the composite 
transmission-rating scale and opinion models. Comparisons of the 
individual test results and the final model are also presented. 

II. SUMMARY OF TRANSMISSION-RATING AND OPINION MODELS 

The models for transmission rating described in this section are 
based on the results of seven subjective tests. All of the tests were 
conducted with Western Electric 500-type telephone sets. 7 Loudness 



* Results of subjective tests in terms of the number of votes in each of the several 
categories of a rating scale can be expressed in a number of ways. One way is to 
assign numerical values to each of the categories, e.g., excellent = 5, good = 4, 
fair = 3, poor = 2, and unsatisfactory = 1 . Each of these numerics is then weighted 
by the proportion of votes in the corresponding category for a particular transmission 
condition, and the weighted values summed. The result is called the mean opinion 
score for that transmission condition. 
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1 — Summary of tests 
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* These tests were not, strictly speaking, two-way conversation tests and, thus, connection loudness- 
loss values are not appropriate. 

loss values used in the model describe the acoustic-to-acoustic transfer 
efficiency of overall telephone connections and are expressed in terms 
of the Electro-Acoustic Rating System (ears) method. 8 Noise values 
used in the model are expressed at the line terminals of a telephone set 
with a reference receiving efficiency of 26 dB based on the ears 
method. * 

The major aspects of these tests are summarized in Table I. A more 
detailed description of the tests is presented in Sections III and IV. 

The results of the subjective tests were used to derive transmission- 
rating models for (i) loss and noise, (ii) talker echo, and (Hi) the com- 
bined effects of loss, noise, and talker echo. In addition, models were 
derived for the relationship between transmission rating and subjective 
opinion. 

The procedures used in the analysis of the subjective-test results 
and the derivation of the transmission-rating scale are described in 
Sections V and VI. Although the procedures are somewhat complex 
for manual calculation, they are easily handled on a digital computer 
and have been found to provide a convenient and useful representation 
for a large variety of test data. 

Mathematical expressions for the models are summarized in Table 
II. The derivations of these expressions are also given in Sections V 



* The several subjective tests, results of which were used in deriving the model, 
were conducted with different circuit-noise values and with telephone sets operating 
at different ears receiving efficiencies. To enable combination of results from the 
different tests to be made, it was necessary to express all circuit-noise values in terms 
of a telephone set with reference receiving sensitivity. The value of 26 dB was chosen 
because it is approximately the receiving efficiency of a customer loop consisting of a 
Western Electric 500- type telephone set, 7 a short-line facility, and a standard central 
office feeding bridge. Noise values are given in dBrnC. 9,10 
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and VI. The remainder of the present section provides a general 
description and graphical presentation of the models. 

2.1 Connection loudness loss and circuit-noise model 

Transmission rating as a function of connection loudness loss and 
message-circuit noise is shown in Fig. 1. The curves were plotted using 

Table II — Models for estimating subjective reaction to loss, 
noise, and echo 

The models, in terms of a transmission-rating scale, for loss and noise (Rln), 
echo (Re), and loss, noise, and echo (Rlne) are: 

R LN = 147.76 - 2.257V (L. - 7.2) J + 1 - 2.009JV* + 0.02037 (L,)Np (1) 

Re = 95.01 - 53.45 log,„( (1 + Z>)/Vl + (D/480) 2 ) + 2.211E (2) 

Rlhb = **£ + ** - ^[ ^"^ J+ooy, (3) 

where L, = Acoustic-to-acoustic loudness loss (in dB) of an overall telephone connec- 
tion, determined using the Electro-Acoustic Rating System (ears) 
method, 
N = Circuit noise (in dBrnC) at the input to a set with a receiving-loudness 
rating of 26 dB, determined using the ears method, 
Nf = Total noise in dBrnC resulting from power addition of the circuit noise, 
N, and 27.37, both in dBrnC, 
D = Round- trip echo-path delay (in milliseconds), and 

E = Acoustic-to-acoustic loudness loss (in dB) of the echo path, determined 
using the ears method. 

The proportion of comments good or better (GoB) or poor or worse (PoW) are com- 
puted from R by : 

GoB = ^L [ A e-'^dt 
V2W-- 



V2^. 

PoW = -rL (' e-*i*dt, 

V2WB 



where A and B are given in the table below for the various data bases. 
Data Base A B 



MH (ft - 64.07)/17.57 (R - 51.87)/17.57 

HOI (R - 77.44)/17.07 (R - 60.70)/17.07 

H02 (R - 73.74)/15.68 (R - 58.03)/15.68 

Echol (« - 75.05)/14.30 (R - 58.95)/14.30 

Echo2 (R - 66.66)/11.84 (R - 53.33)/11.84 

The parameters A and B have been derived from opinion distributions, in terms of 
fit mean, p, and fit standard deviation, a, and then expressed as a function of R. 
Alternatively, the models can be expressed as follows: 

mn = (R - 21.37)/12.20 <r a = 1-44 

Mhoi = (mmh -I- 0.206)/1.372 = (R - 18.86)/16.74 a a = 1.02 

/xho2 = (pmh + 0.215)/1.288 = (R - 18.75)/15.71 a a = 0.998 

mei - (22 - 18.7)/16.1 <r a = 0.888 

mes = (R - 20)/13.33 a a = 0.888 
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Fig. 1 — Transmission rating for loss and noise. 



eq. (1) below, which is also given in Table II. 

R LN = 147.76 - 2.257V (L. - 7.2) 2 + 1 - 2.009iV> 

+ 0.02037 (L e )N F , (1) 

where 

L e = Acoustic-to-acoustic loudness loss (in dB) of an overall 

telephone connection 
N = Circuit noise (in dBrnC) at the input to a set with a receiving 

loudness rating of 26 dB 
N F = Power addition of the circuit noise, N, and 27.37 dBrnC. 

The transmission-rating scale was derived, so that it is anchored at 
two points, as shown in Table III. These anchor points were selected 
to be well separated in quality, but within the range of conditions 
that are likely to be included in a test. Transmission ratings for other 
combinations of connection loudness loss and circuit noise are relative 
to those for the two anchor points. The rating values are such that 
most telephone connections will have positive ratings between 40 and 
100, with the higher rating denoting higher quality. For most engineer- 
ing applications sufficient accuracy can be achieved by the use of 
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Table III — Anchor conditions for the transmission-rating scale 



Connection Loudness 
Loss 
(dB) 


Circuit Noise 
(dBrnC) 


Transmission 
Rating 


15 
30 


25 
40 


80 
40 



whole numbers on the transmission-rating scale. For example, in the 
1965 loss and noise tests, conditions with a transmission rating of 
approximately 80 were considered good or excellent by 80 percent of 
the subjects, while a transmission rating of 40 was considered good or 
excellent by only 10 percent. 

2.2 Talker-echo model 

Transmission rating as a function of talker-echo path loss and delay 
is shown in Fig. 2. The curves were plotted using eq. (2) below, which 
is also given in Table II. This equation was derived to exclude the 
effects of circuit noise and connection loudness loss. 



Re = 95.01 - 53.45 log 10 {[1 + D]/Vl + (Z)/480) 2 } + 2.277tf, (2) 




10 20 30 40 50 

ECHO-PATH LOUDNESS LOSS, E, IN dB 

Fig. 2 — Transmission rating for talker echo. 
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where 

D — echo-path delay (in ms) and 

E = acoustic-to-acoustic loudness loss (in dB) of the echo path. 

The curves of Fig. 2 demonstrate the dependence of transmission 
quality on the two talker-echo path parameters, loss and delay, for 
connections where talker echo is important. 

2.3 Connection loudness loss, circuit noise, and talker-echo model 

Transmission ratings for the combined effects of connection loudness 
loss, circuit noise, echo-path loudness loss and echo-path delay are 
obtained from eq. (3) below, which is also given in Table II. 



Rlne — 



Rln + Re 



where 



_^ fi L ,-^ y + (10)2> 



(3) 



Rlne = transmission rating for the combined effects of connection 
loudness loss, circuit noise, and talker echo, 
Rln = transmission rating for connection loudness loss and circuit 
noise 
Re = transmission rating for echo-path loudness loss and delay. 



100 



90 



BO 



CONNECTION LOUDNESS LOSS, L E -15dB 
CIRCUIT NOISE, N = 30dBrnC 




10 20 30 40 50 60 

ECHO-PATH LOUDNESS LOSS, E, IN dB 



Fig. 3 — Transmission rating for loss, noise, and talker echo. 
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Figure 3 illustrates curves generated by means of the above relation- 
ship for the transmission rating as a function of echo-path loudness 
loss and delay in a connection with a connection loudness loss of 15 dB 
and circuit noise of 30 dBrnC. For other values of connection loudness 
loss and circuit noise, the curves would become asymptotic to higher or 
lower values of R in accordance with the curves of Fig. 1. 

2.4 Subjective-opinion models 

Subjective opinion in terms of the proportion of ratings in each of 
the five categories (E, G, F, P, U) for a condition having a given 
transmission rating has been found to depend on various factors, such 
as the subject group, the range of conditions presented in a test, the 
year in which the test was conducted, and whether the test was con- 
ducted on conversations in a laboratory environment or on normal 
telephone calls. For the major tests on which the transmission-rating 
model is based, the observed relationship between subjective judgments 
and transmission rating can be represented as shown in Figs. 4 and 5 
which are plotted from the equations for GoB and PoW of Table II. 
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Fig. 4 — Subjective opinion as a function of transmission rating for the connection 
loudness loss and circuit-noise tests. 
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Fig. 5 — Subjective opinion as a function of transmission rating for the echo tests. 

Contours of constant percent good or better and percent poor or 
worse are given in Figs. 6 and 7, respectively, for the loss-noise results. 
These contours were computed from eq. (1) and the equations for 
GoB and PoW of Table II using the MH data base. Similar contours 
could be generated based on one of the other tests. However, the 
Murray Hill base is being used for current network planning studies 
for consistency with earlier studies. In addition, the opinion results 
from this test appear to be in close agreement with data obtained from 
customer interviews on typical ddd toll connections. Thus, at the time 
of publication, the Murray Hill base is recommended for conversion 
of the transmission ratings to subjective ratings. Eventually, we hope 
to determine other expressions for A and B (see Table II) that will 
provide even better agreement with customer interviews on various 
types of telephone connections. 

2.5 Use of the models 

The models summarized in preceding sections can be used to esti- 
mate transmission quality for telephone connections. The examples 
given below are based on representative 500-type telephone sets con- 
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Fig. 6 — Subjective-opinion contours of percent good or better at the MH base for 
loss and noise. 
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nected by specified lengths of 26-gauge nonloaded cable to the local 
Class 5 office. 1 The loss for such an arrangement is expressed in terms of : 

tlr = Transmitting loop rating (in dB), which describes the loud- 
ness conversion efficiency in terms of an acoustic signal 
applied at the telephone set and the resulting electric voltage 
at the local Class 5 office. 

rlr = Receiving loop rating (in dB), which describes the loudness- 
conversion efficiency in terms of an electric circuit voltage 
applied at the local Class 5 office and the resulting acoustic 
pressure received at the telephone set. 8 

The connection loudness loss, L e , for a local call is the sum of tlr and 
rlr. For calls between customers served by different local Class 5 
offices, L e is the sum (tlr + rlr) plus the 1000-Hz loss between the 
offices. 

The circuit noise, N (in dBrnC), as used in the model is in terms 
of a reference telephone set that has an rlr of 26 dB. Noise levels, as 
typically expressed at the line terminals of a telephone set, need to 
be corrected to the reference set. For the 500-type telephone set, the 
conversion factor is about 4 dB (increase in noise) and is nearly in- 
dependent of loop length. 

The effect of talker echo depends on the characteristics of the echo 
path. Generally, the dominating path is that from the talking customer 
to the distant Class 5 office and return, and is referred to as the far- 
end echo path. 11 The round-trip delay of this path, D, is taken to be 
the time required for a speech signal to go from the originating Class 5 
office to the distant Class 5 office and return. (Delay in the customer's 
telephone set and loop is neglected as, with present plant, it is usually 
insignificant.) The loudness loss of the echo path, E, is the sum of 
tlr, rlr, and the echo-path loss from the originating Class 5 office to 
the distant Class 5 office and return. 

Considerations of the preceding paragraphs provide the basis for 
several examples demonstrating use of the models. Details concerning 
computation for these examples are given in Appendix A. Results of 
these examples are summarized in Table IV. The examples are simpli- 
fied representations of connections devised to illustrate application of 
the models and, thus, the results only approximately describe the 
performance of actual connections. (Methods of obtaining more accu- 
rate connection representations are covered in Ref. 1.) 

Comparison of the results for Examples 1, 2, and 3 show that about 
optimum performance for local connections occurs for medium loops. 
The performance is below optimum for short and long loops, the former 
because the loss is lower, the latter because the loss is higher (see 
Figs. 1, 6, and 7). 
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Table IV — Examples of performance estimates obtained 
using the models 



Example 


(dB) 


N 

(dBrnC) 


D 

(ms) 


E 

(dB) 


Rln 


R. 


Rlns 


GoB 

(%) 


PoW 

(%) 


1. Local connection, 




















short loops 


2.7 


27 
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78.3 


— 
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79.2 


6.6 


2. Local connection, 




















medium loops 


8.4 


27 
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88.7 


— 


— 


92 


1.8 


3. Local connection, 
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75.5 
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74.2 


8.9 


4. Long-toll connection, 
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65.2 


13.9 


5. Long-toll connection, 




















medium loops, with 




















talker echo 


16.1 


32.5 


37.3 


31.7 


71 


82.6 


65.2 


52.6 


22.4 



Comparison of Examples 3 and 4 illustrates the effect of higher loss 
and noise typically encountered on toll connections. 12 Finally, compari- 
son of Examples 4 and 5 indicates the effect of talker echo, demon- 
strating the need for echo control. (See Ref. 1 for detailed discussion 
of loss, noise, and talker echo for toll connections.) 

III. DESCRIPTION OF LOSS-NOISE SUBJECTIVE TESTS 

Three tests have been conducted using a special test facility called 
sibyl to determine subjective reaction to loss and noise on telephone 
connections. This facility allowed control of transmission parameters 
during normal business calls of cooperating Bell Laboratories 
employees. 13-15 

sibyl was first used at the Murray Hill Bell Laboratories location, 
and was moved to the Holmdel location in 1966. 

The test subjects for the sibyl studies were Bell Laboratories em- 
ployees. Prior to the beginning of each of the tests, a list of employees 
reflecting a preselected makeup of age, sex, etc., was obtained, and the 
employees contacted to solicit their participation in the test. Upon 
obtaining agreement, their telephone lines were routed through sibyl 
which could handle up to 100 subject lines. 

An overall connection with sibyl inserted is shown on Fig. 8. At 
the left is a subject's (participating employee's) telephone set. In 
close physical proximity (less than about 1500 feet of two-conductor 
cable) is sibyl which converts the two-wire transmission path into a 
four-wire transmission path and separates signals transmitted from 
the subject's telephone set and signals received at the same telephone 
set. Separation of the signal paths in this manner permits (t) inserting 
different impairment values for the two directions of transmission and 
(ii) independent measurement of signals transmitted from and received 
at the subject's telephone set. 

Proceeding from left to right in Fig. 8, the four-wire path is re- 
converted to a two-wire path in sibyl, and connected to the serving 
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Fig. 8 — Diagram of a telephone connection for the sibyl tests. 

central office over a two-wire cable pair which is about 1 mile in length 
for Murray Hill and about 3 miles in length for Holmdel. The central 
office switching machine connects the subject's line to the called line 
dialed by the subject. The called line (approximate lengths as given 
above for Murray Hill and Holmdel) is terminated with the telephone 
set of the called party, another Bell Laboratories employee at the 
same location who is usually not a subject in the experiment. 

siybl recognized calls internal to the location, and only such calls 
were included in the experiments reported here. The major reason for 
this was to retain as much control as possible of the transmission 
parameters on test calls. Employees' telephone lines at any one location 
generally were of about the same physical makeup. Thus, variations 
in transmission parameter values between lines were small, and the 
values were considered to be identical for all lines, sibyl then altered 
the normal parameter values to achieve the parameter values of interest 
in an experiment. 

Restricting the calls to within the location had the further advantage 
that the subjects physical environment was reasonably uniform, con- 
sisting largely of offices with one to four desks and of electronic labora- 
tories. Thus, room noise levels, which could affect the transmission 
quality of telephone calls, did not vary appreciably. 

The subjects were provided with instructions at the beginning of the 
test, and thereafter, the procedure during a call was generally as shown 
in Fig. 9. (The procedure of Fig. 9 applies specifically to the Murray 
Hill test.) The subject initiated the procedure by lifting his handset 
from the telephone set cradle and dialing a number. If the call was 
not a test call, which was the case for about 85 percent of the calls 
from the subject group, the call was completed normally. That is, 
(i) the subject received a signal indicating the called line was busy 
and returned his handset to the cradle, (it) the called party did not 
answer and the subject returned his handset to the cradle, or (in) 
the called party answered and they conducted their conversation, 
after which the subject and called party returned their respective 
handsets to the cradles. 



1332 THE BELL SYSTEM TECHNICAL JOURNAL, NOVEMBER 1976 



If the call was a test call and the called line was not busy, a test 
condition (e.g., a predetermined combination of loss and noise values) 
was inserted at the beginning of the call, and the conversation pro- 
ceeded. If, during the call, the test condition was unacceptable to the 
subject (e.g., the noise was too loud) the subject could dial a digit 
(e.g., a five) on his telephone set dial. This signaled sibyl to remove 
the test condition, after which the call completed normally. (This 
occurred on only about 1 to 2 percent of all test calls.) 

In the more usual case, the subject was either not consciously aware 
of the test condition or did not find it unacceptable and completed the 
conversation, then returned the handset to the cradle. A short time 
thereafter, typically seconds, the subject received ringback — the 
telephone set ringer emitted a burst of sound — which alerted the 
subject to rate the transmission quality of the call just completed. 

3.1 1965 Murray Hill SIBYL test 

In 1965, a test to determine subjective reaction to loss and noise 
was conducted using the sibyl facility, which was then located at Bell 
Laboratories, Murray Hill, New Jersey. The configuration of a typical 
connection incorporating sibyl is shown on Fig. 8 as discussed earlier. 
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Fig. 9 — Procedure for a sibyl call. 
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Table V — 1965 Murray Hill sibyl test of loss and noise: 
number of ratings for each test condition 



Connection 




Circuit Noise (dBrnC) 




Loudness Loss 
(dB) 








21 


28 


36 


44 


5 


35 


12 


24 


20 


10 


61 


36 


34 


34 


15 


33 


33 


27 


28 


20 


29 


43 


23 


35 


25 


35 


32 


29 


24 


30 


12 


12 


23 


11 



The 24 combinations of loss and noise values that were tested are 
given in Table V together with the number of subjective ratings ob- 
tained for each combination. The loss values of Table V represent 
acoustic-to-acoustic loudness loss (in dB) of the connections, and are 
numerically equivalent to the electrical losses in terms of which results 
were published earlier for this test. 6 Circuit noise levels (in dBrnC) 
were originally reported in terms of (i) added noise, and did not include 
allowance for noise normally present on the connections, and (it) the 
average receiving sensitivity of Western Electric 500-type telephone 
sets at the Murray Hill location. The noise levels of Table V represent 
total noise from all sources and are expressed at the line terminals of a 
telephone set with reference receiving sensitivity. The condition with 
loudness loss = 5 dB and noise = 21 dBrnC represents average trans- 
mission normally experienced by employees on calls within the Murray 
Hill location. 

The lines connecting employees' telephone sets to the central office 
were all of about the same length. Thus, the loudness loss of the 
telephone side tone path, 16 ' 17 strongly dependent on line length, was 
expected to be about the same for all employees. Side tone path loud- 
ness loss was measured on a sample of lines and telephone sets, and 
found to be about 13 dB. Circuitry was incorporated into sibyl to 
ensure that the side tone path loudness loss would also be about 13 dB 
for subjects' telephone sets when connected to sibyl. 

Room noise was measured at a sample of subject locations using a 
sound-level meter with A-weighting. 18 The average value was found to 
be 45 dB(A) with a range of ±2 dB. 

Sixty-six employees (subjects) participated in the test. The pre- 
determined combinations of loss and noise were randomly introduced 
into about 15 percent of all within-location, normal-business calls 
placed by the subject group during the 3-month test interval. A subject 
had no prior information that any particular call was being placed 
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over a test connection. The call procedure summarized in Fig. 9 was 
followed for each call. At the end of each test call, a subject judged 
overall transmission quality by dialing a 9 for excellent, 8 for good, 7 
for fair, and 6 for poor. Connections rejected during the call were 
considered to be unsatisfactory. 

Test results are given in Table VI for each test condition in terms of 
the percent of subjects' votes in each of the five rating categories. These 
results are used in Sections V and VI in deriving the subjective-opinion 
models. 

3.2 1969 Holmdel 1 SIBYL test 

As discussed earlier, a second test to determine subjective reaction 
to loss and noise was conducted in 1969. This test utilized the sibyl 
facility which was moved to the Holmdel location of Bell Laboratories 
in 1966. A major reason for these newer tests was to examine the effect 
of asymmetric transmission conditions on subjects' ratings. However, 
the test included 15 symmetric conditions (out of a total of 47 test 
conditions) and these are considered in this paper. 

Table VI — 1965 Murray Hill sibyl test of loss and noise: 

test results 



Circuit 


Connection 


Percent of Subjects' Votes in Each Category 


Noise 


Loudness Loss 
















(dBrnC) 


(dB) 


Excel. 


Good 


Fair 


Poor 


Unsat. 


21 


5 


60.0 


11.4 


14.3 


14.3 







10 


63.9 


26.2 


3.3 


6.6 







15 


66.7 


21.2 


12.1 










20 


41.3 


27.6 


24.1 


3.5 


3.5 




25 


34.3 


14.3 


31.4 


8.6 


11.4 




30 


8.3 


25 


33.3 


16.7 


16.7 


28 


5 


50.0 


33.3 


16.7 










10 


66.7 


30.5 





2.8 







15 


54.5 


30.3 


9.1 


6.1 







20 


25.6 


30.2 


18.6 


16.3 


9.3 




25 


25.0 


15.6 


37.5 


12.5 


9.4 




30 


8.3 


8.3 


33.3 


16.8 


33.3 


36 


5 


54.2 


20.8 


16.7 


8.3 







10 


64.7 


23.6 


5.9 


2.9 


2.9 




15 


33.3 


29.6 


22.2 


14.9 







20 


21.7 


8.7 


34.8 


21.7 


13.1 




25 


13.8 


3.5 


17.2 


34.5 


31.0 




30 





13.0 


26.1 


34.8 


26.1 


44 


5 


15.0 


30.0 


25.0 


20.0 


10.0 




10 


17.6 


26.5 


11.8 


20.6 


23.5 




15 


17.8 


25.0 


28.6 


14.3 


14.3 




20 


8.6 


5.7 


20.0 


40.0 


25.7 




25 





4.2 


25.0 


50.0 


20.8 




30 











36.4 


63.6 
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Table VII — 1969 Holmdel 1 sibyl test of loss and noise: 
number of ratings for each test condition 



Connection 




Circuit Noise (dBrnC) 




Loudness Loss 
(dB) 








22 


35 


40 


45 


10 


335 


48 


41 


38 


15 


93 


— 


— 


— 


20 


65 


57 


77 


77 


25 


33 


34 


47 


— 


30 


76 


61 


— 


81 



The symmetric combinations of loss and noise values that were 
tested are given in Table VII together with the number of subjective 
ratings obtained for each condition. The condition with connection 
loudness loss = 10 dB and circuit noise = 25 dBrnC was the reference 
condition that was repeated frequently during the test. 

Side tone path loudness loss was measured on a sample of lines and 
telephone sets, and found to average about 12 dB with a range of 
±2 dB both for normal connections and sibyl connections. Room 
noise, measured at 12 subject locations, averaged about 42 dB(A) 
with a range of about ±2 dB. 

Seventy-eight employees (subjects) participated in this test. Sub- 
jects followed the same procedure during the 2\ month test interval 
as already described for the Murray Hill test. Comparison of the 
Holmdel 1 test results of Table VIII with the Murray Hill test results 
of Table VI shows that in the former, subjects gave lower ratings 
for approximately equivalent combinations of loss and noise than is 
the case for the latter. The major differences between the two tests 
were (i) the subject groups, (ii) the time difference of about 4 years, 
and (iii) the location. While it is not clear how these differences con- 
tributed to differences in results, the comparison suggests that sub- 
jects' expectations were higher in the later Holmdel tests. 

3.3 1972 Holmdel 2 SIBYL test 

A third test to determine subjective reaction to loss and noise was 
conducted in 1972. This test also utilized the sibyl facility at the 
Holmdel location of Bell Laboratories. 

A major purpose of the test was to determine whether or not the 
more critical subjective evaluations found in the Holmdel 1 test would 
continue to hold. The test included combinations of crosstalk, loss, 
and noise as well as combinations of loss and noise. Only the latter are 
considered here because only the results for these conditions are used 
in the subjective-opinion model for loss and noise. 
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Table VIII — 1969 Holmdel 1 sibyl test of loss and noise: 

test results 



Circuit 


Connection 


Percent of Subjects' Votes in Each Category 


Noise 


Loudness Loss 
















(dBrnC) 


(dB) 


Excel. 


Good 


Fair 


Poor 


Unsat. 


22 


10 


37.9 


41.2 


13.7 


4.5 


2.7 




15 


29.0 


40.9 


21.5 


5.4 


3.2 




20 


15.4 


26.1 


43.1 


7.7 


7.7 




25 


3.0 


15.1 


27.3 


27.3 


27.3 




30 


2.6 


13.2 


34.2 


30.3 


19.7 


35 


10 


12.5 


20.8 


39.6 


25.0 


2.1 




20 


1.8 


10.5 


45.6 


26.3 


15.8 




25 





2.9 


20.6 


53.0 


23.5 




30 


1.6 


3.3 


16.4 


47.5 


31.2 


40 


10 


4.9 


34.1 


24.4 


26.8 


9.8 




20 





1.3 


14.3 


53.2 


31.2 




25 


2.1 





12.8 


51.1 


34.0 


45 


10 





7.9 


21.0 


47.4 


23.7 




20 


1.3 





22.1 


39.0 


37.6 




30 








1.2 


33.3 


65.5 



The 12 symmetric combinations of loss and noise values tested are 
given in Table IX together with the number of subjective ratings 
obtained for each combination. These conditions covered about the 
same range as for the Holmdel 1 (HOI) sibyl test, except that a 
condition of lower connection loudness loss of 5 dB was included in 
the Holmdel 2 (H02) sibyl test to match the range of loss in the 
Murray Hill (MH) test. 

Room-noise levels and side tone path loudness loss values were 
assumed to be identical with those for the HOI test. 

Seventy-four employees (subjects) participated in the test. (None 
of these employees had been subjects in the HOI test.) The subjects 
followed the same procedure as has already been described for the MH 

Table IX — 1972 Holmdel 2 sibyl test of loss and noise: 
number of ratings for each test condition 



Connection 


Circuit Noise (dBrnC) 


Loudness Loss 
(dB) 








25 


32 


42 


5 


99 


70 


53 


10 


1029 


58 


97 


20 


50 


64 


46 


30 


62 


52 


4 
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Table X — 1972 Holmdel 2 sibyl test of loss and noise: 

test results 



Circuit 


Connection 


Percent of Subjects' Votes in Each Category 


Noise 


Loudness Loss 


















(dBrnC) 


(dB) 


Excel. 


Good 


Fair 


Poor 


Unsat. 


25 


5 


49.5 


35.4 


13.1 





2.0 




10 


47.2 


40.6 


10.2 


1.3 


0.7 




20 


14.0 


26.0 


40.0 


18.0 


2.0 




30 


3.2 


3.2 


32.3 


35.5 


25.8 


32 


5 


27.1 


35.7 


31.4 


5.8 







10 


19.0 


39.6 


32.8 


6.9 


1.7 




20 


4.7 


6.3 


48.3 


34.4 


6.3 




30 


1.9 


7.7 


15.4 


51.9 


23.1 


42 


5 


11.3 


11.3 


35.8 


34.0 


7.6 




10 


17.5 


17.5 


33.1 


24.7 


7.2 




20 


4.3 


10.9 


19.6 


41.3 


23.9 




30 








25.0 





75.0 



and HOI tests, except that the voting procedure was changed. In 
the H02 test, subjects were instructed that at the end of each experi- 
mental call, when they received ringback, they should rate the overall 
transmission quality of the call by dialing 9 for excellent, 8 for good, 
7 for fair, 6 for poor, and 5 for unsatisfactory. In addition, they could 
reject an unacceptable connection during a call by dialing 4. In the 
latter case, they still received ringback after the call and were asked 
to rate the quality according to the 5-point scale. (In the MH and HOI 
tests, a 4-point scale was used for post-call rating, and the fifth point) 
unsatisfactory, was assumed for dialed-out calls.) 

Results of the H02 test are given in Table X. These results are in 
close agreement with those of the HOI test. Comparison of the test 
results for the three loss-noise tests are dealt with further in later 
sections covering derivation of the subjective-opinion models. 

IV. DESCRIPTION OF TALKER-ECHO SUBJECTIVE TESTS 

Talker echo occurs on a telephone connection when a portion of the 
primary speech signal is reflected at an impedance mismatch at some 
point in the connection, and returned to the talker delayed in time. 
The returned signal, talker echo, is defined in terms of echo-path 
delay and echo-path loudness loss. The echo-path delay that occurs 
because of the finite propagation velocity of the speech signal over 
transmission facilities and equipments is the time it takes the speech 
signal to traverse the path from the talker's lips to the point of im- 
pedance mismatch, then back to the talker's ear. Echo-path loudness 
loss represents the amount by which the talker's speech signal is 
attenuated when traversing the same path. 
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Four tests were conducted to determine subjective reaction to 
talker echo. Three of these tests — identified as the 1966 Laboratory 
Echo Test, the 1968 Laboratory Echo Test, and the 1970 Laboratory 
Echo Test — were conducted under laboratory conditions where the 
experimenter could closely control conditions. The laboratories used 
in these tests included rooms that were acoustically designed to muffle 
both internal and external noise. 15 The fourth test, identified as the 
1970 sibyl Echo Test, was conducted using the sibyl facility. 

The 1966 and 1968 Laboratory Echo Tests and the 1970 sibyl 
Echo Tests were designed to study subjective reaction at short echo 
delays ( < 100 ms) such as might be encountered on long terrestrial 
connections. The 1970 Laboratory Echo Test considered the effects of 
long delays that might be encountered on connections using one- and 
two-hop synchronous-orbit satellite connections. 

4.1 1966 laboratory echo test 

The test conducted in 1966 to determine subjective reaction to 
talker echo utilized the test system shown in block diagram form on 
Fig. 10. This system provided (i) a fixed sidetone path with a loudness 
loss of about 9 dB, (ii) an echo path by means of which the subject 
heard his own voice delayed in time and attenuated under control of 
the experimenter, and (in) a transmission path from the test adminis- 
trator to the subject that had a loudness loss of 14 dB. (Transmission 
from the subject to the test administrator was obtained by means of 
an intercom system.) The administrator and subject were located in 
separate rooms for which the ambient room noise was about 35 dB(A), 
presumably sufficiently low so as to not affect subjects' ratings. 
Circuit noise was held constant at 28 dBrnC. 

The subject was first given four practice conditions to illustrate the 
range of transmission quality. Then the actual test conditions were 
presented. The test incorporated 30 conditions, five different values of 
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Fig. 10 — Test system for the 1966 and 1968 laboratory echo tests. 
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echo-path delay each with six different values of echo-path loudness 
loss. These conditions were presented in random order and each 
subject evaluated each condition once. 

At the beginning of each test session, a subject was seated in the 
test room and given general instructions for the test by the adminis- 
trator speaking over the system of Fig. 10. The administrator's talking 
level was held constant. For each of the selected test conditions, the 
administrator spoke phonetically balanced sentences with the subject 
repeating each sentence immediately afterward. This continued until 
the subject arrived at a rating (excellent, good, fair, poor, un- 
satisfactory) for the test condition. Then the next condition was 
administered. 

The test conditions and the results obtained for the 29-member 
subject group are given in Table XI. These results show that for any 
given echo-path delay, the transmission quality improves with in- 
creasing echo-path loudness loss. Also, the data indicate that for any 
given echo-path loudness loss, the transmission quality is degraded 
with increasing echo-path delay. 

4.2 1968 laboratory echo test 

This test, conducted in 1968, was designed on the basis of results 
obtained in the 1966 Laboratory Echo Test. The test system was the 
same as discussed in the preceding section. 

Side tone path loudness loss was constant at 12 dB. Ambient room 
noise was about 35 dB(A). 

The test incorporated 93 conditions. Three of these were base con- 
ditions at three different circuit-noise levels. The remaining 90 condi- 
tions represented various combinations of circuit-noise level, echo-path 
delay, and echo-path loudness loss. These conditions were arranged in 
random order and presented to each test subject in two sessions to 
avoid subject fatigue. 

The procedure for each subject followed that discussed in Section 

4.2 except that only three practice conditions were used. 

The test conditions and the results obtained for the 30-member 
subject group are given in Table XII. These test results are reported 
in part 1 of Annex 4 to Question 6/XII in Ref. 19. 

4.3 1970 laboratory echo test 

Tests to determine subjective reaction to echo on circuits with echo- 
path delays of ms, 65 ms, 600 ms, and 1200 ms were conducted in 
1970. The tests consisted of a total of 25 conditions, many of which 
included echo suppressors. One condition at ms, five conditions at 
600 ms, and four conditions at 1200 ms did not employ echo sup- 
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Table XI — 1966 laboratory echo-test results 




Echo Path 


Percent of Subjects' Votes in Each Category 


Delay 
(ms) 


Loudness Loss 
(dB) 


Excel. 


Good 


Fair 


Poor 


Unsat. 


20 


10 


3.5 


3.5 


41.3 


37.9 


13.8 




15 





10.3 


48.3 


37.9 


3.5 




20 


6.9 


27.6 


51.7 


13.8 







25 


41.4 


48.3 


10.3 










30 


44.8 


48.3 


6.9 










35 


51.6 


44.9 





3.5 





36 


16.5 


3.5 


3.5 


24.0 


55.2 


13.8 




21.5 


3.5 


20.7 


20.7 


51.6 


3.5 




26.5 


10.3 


58.6 


27.6 


3.5 







31.5 


20.7 


58.6 


17.2 


3.5 







36.5 


41.3 


51.7 


3.5 


3.5 







41.5 


62.0 


31.0 


3.5 


3.5 





56 


26 


3.5 


3.5 


55.1 


34.4 


3.5 




31 


10.3 


17.3 


51.7 


20.7 







36 


10.3 


34.5 


51.7 


3.5 







41 


24.1 


48.3 


27.6 










46 


51.7 


41.3 


3.5 





3.5 




51 


55.2 


41.3 








3.5 


72 


29 


3.5 


6.9 


37.9 


51.7 







34 





31.0 


51.7 


17.2 







39 





48.3 


48.2 


3.5 







44 


20.7 


51.7 


27.6 










49 


41.4 


37.9 


20.7 










54 


44.7 


48.3 


3.5 


3.5 





90 


34 





10.3 


48.3 


37.9 


3.5 




39 


6.9 


31.0 


48.3 


10.3 


3.5 




44 


10.3 


48.3 


41.4 










49 


24.1 


58.7 


10.3 


6.9 







54 


41.4 


44.8 


13.8 










59 


55.2 


41.3 


3.5 









pressors. Results for these conditions were used in deriving the echo 
model. 

The test system, shown on Fig. 11, provided for two-way conversa- 
tion between pairs of subjects. The two ends of the test system were 
located in separate, acoustically treated rooms for which the ambient 
room noise was about 38 dB(A). Each subject was able to hear his 
own voice by means of (i) a side tone path with loudness loss of 12 dB 
and (ii) an echo path. 

One hundred pairs of subjects participated in the tests; 50 of these 
evaluated the 600-ms delay case, the other 50 evaluated the 1200-ms 
delay case. The two members of a pair (they were acquainted) were 
located in the two separate test rooms. Prior to a test session, they 
were instructed that they should discuss a subject of mutual interest 
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Table XII — 1968 laboratory echo-test results — 30 ratings 
(subjects) per condition 





Echo Path 


Percent of Subjects' Votes 


in Each Category 


Circuit 

Noise 

(dBrnC) 












Delay 

(ms) 


Loudness 
Loss 


Excel. 


Good 


Fair 


Poor 


Unsat. 




(dB) 












18.0 


— 


100 


63.4 


30.0 


3.3 





3.3 


18.0 


1.5 


0.4 


3.3 


33.3 


36.7 


23.4 


3.3 






5.4 


26.7 


43.3 


23.3 


6.7 









10.4 


33.3 


46.7 


10.0 


10.0 









15.4 


60.0 


26.7 


13.3 












20.4 


33.3 


46.7 


13.3 


6.7 









25.4 


63.3 


36.7 











18.0 


20.0 


5.5 





3.3 


3.3 


23.4 


70.0 






10.5 


3.3 





13.3 


40.0 


43.4 






15.5 





10.0 


36.7 


23.3 


30.0 






20.5 


33.3 


30.0 


16.7 


13.3 


6.7 






25.5 


36.7 


36.7 


23.3 


3.3 









30.5 


26.7 


56.7 


6.6 


10.0 





18.0 


56.0 


15.6 








6.7 


33.3 


60.0 






20.6 








10.0 


36.7 


53.3 






25.6 


3.3 


6.7 


26.7 


36.6 


26.7 






30.6 


3.3 


20.0 


40.0 


33.4 


3.3 






35.6 


20.0 


43.3 


26.7 


10.0 









40.6 


33.3 


46.7 


16.7 





3.3 


18.0 


90.0 


24.5 








6.7 


40.0 


53.3 






29.5 


3.3 


10.0 


20.0 


36.7 


30.0 






34.5 


3.3 


13.3 


43.3 


36.8 


3.3 






39.5 


23.3 


26.7 


16.7 


30.0 


3.3 






44.5 


30.0 


40.0 


23.4 


3.3 


3.3 






49.5 


50.0 


40.0 


10.0 








28.0 


— 


100 


26.6 


60.0 


6.7 


6.7 





28.0 


1.5 


0.4 


3.3 


26.8 


43.3 


13.3 


13.3 






5.4 


10.0 


43.3 


40.0 


6.7 









10.4 


16.7 


56.7 


20.0 


6.6 









15.4 


16.7 


56.7 


16.7 


6.6 


3.3 






20.4 


6.7 


56.7 


30.0 


3.3 


3.3 






25.4 


23.3 


60.0 


13.4 





3.3 


28.0 


10.0 


3.0 





3.4 


23.3 


33.3 


40.0 






8.0 





3.3 


43.3 


36.7 


16.7 






13.0 


3.3 


20.0 


33.3 


26.7 


16.7 






18.0 


20.0 


56.7 


13.3 


3.3 


6.7 






23.0 


3.3 


46.7 


43.3 


6.7 









28.0 


10.0 


60.0 


26.7 


3.3 





28.0 


20.0 


5.5 








13.3 


33.3 


53.4 






10.5 


3.3 


6.7 


33.3 


26.7 


30.0 






15.5 


10.0 


3.3 


23.3 


43.4 


20.0 






20.5 


3.3 


23.4 


60.0 


10.0 


3.3 






25.5 


13.3 


56.7 


23.3 





6.7 






30.5 


10.0 


56.7 


23.3 


6.7 


3.3 
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Table XII — Continued 





Echo Path 


Percent of Subjects' Votes 


xi Each Category 


Circuit 

Noise 
(dBrnC) 














Delay 
(ms) 


Loudness 
Loss 

(dB) 


Excel. 


Good 


Fair 


Poor 


Unsat. 


28.0 


36.0 


11.0 








10.0 


16.7 


73.3 






16.0 








6.7 


33.3 


60.0 






21.0 








26.7 


50.0 


23.3 






26.0 


3.3 


20.0 


60.0 


16.7 









31.0 


6.7 


50.0 


33.3 


6.7 


3.3 






36.0 


13.3 


60.0 


20.0 


6.7 







56.0 


15.6 





3.3 


10.0 


20.0 


66.7 






20.6 


3.3 





6.7 


36.7 


53.3 






25.6 





3.3 


40.0 


30.0 


26.7 






30.6 





13.3 


50.0 


33.4 


3.3 






35.6 


3.3 


46.7 


36.7 


13.3 









40.6 


16.7 


50.0 


33.3 










72.0 


21.5 


3.3 


3.3 


6.7 


33.3 


53.4 






26.5 








13.3 


63.4 


23.3 






31.5 


3.3 


3.3 


33.4 


40.0 


20.0 






36.5 


6.7 


36.6 


43.3 


6.7 


6.7 






41.5 


6.7 


36.6 


46.7 


10.0 









46.5 


23.3 


56.7 


13.3 


6.7 







90.0 


24.5 








3.3 


33.3 


63.4 






29.5 


3.3 





26.7 


50.0 


20.0 






34.5 


6.7 


23.4 


33.3 


33.3 


3.3 






39.5 


6.7 


30.0 


46.6 


16.7 









44.5 


3.3 


46.7 


43.3 


6.7 









49.5 


10.0 


50.0 


36.7 





3.3 


38.0 


— 


100 





26.7 


53.3 


13.3 


6.7 


38.0 


1.5 


0.4 


3.3 





26.7 


46.7 


23.3 






5.4 





3.3 


30.0 


53.4 


13.3 






10.4 





26.7 


50.0 


13.3 


10.0 






15.4 


3.3 


26.7 


50.0 


20.0 









20.4 





13.3 


56.7 


20.0 


10.0 






25.4 


6.7 


10.0 


40.0 


40.0 


3.3 




20.0 


5.5 








3.3 


20.0 


76.7 






10.5 








3.3 


36.7 


60.0 






15.5 





3.3 


20.0 


43.4 


33.3 






20.5 





6.7 


43.3 


43.3 


6.7 






25.5 


6.7 


13.3 


46.7 


23.3 


10.0 






30.5 





6.7 


60.0 


20.0 


13.3 




56.0 


15.6 








3.3 


26.7 


70.0 






20.6 








3.3 


46.7 


50.0 






25.6 








20.0 


53.3 


26.7 






30.6 








53.4 


43.3 


3.3 






35.6 


6.7 


16.7 


40.0 


33.3 


3.3 






40.6 





13.3 


46.7 


30.0 


10.0 




90.0 


24.5 








13.3 


36.7 


50.0 






29.5 








16.7 


53.3 


30.0 






34.5 





13.3 


36.7 


43.3 


6.7 






39.5 





23.3 


40.0 


36.7 









44.5 


6.7 


26.7 


40.0 


23.3 


3.3 






49.5 


3.3 


23.3 


56.7 


10.0 


6.7 
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TRANSMITTER 



RECEIVER 



RECEIVER 




TRANSMITTER 



Fig. 11 — Test system for the 1970 laboratory echo test. 



over the system and told that each test condition would require about 
4 minutes of conversation, at the end of which they should separately 
rate the condition on the 5-point scale : excellent, good, fair, poor, and 
bad. (These categories were further subdivided into undesignated 
thirds, resulting in a 1 5-point scale.) 

The test conditions and results are given in Table XIII and are also 
reported in Part III of Annex 5 to Question 6/XII in Ref. 19. As with 
tests discussed earlier, these results show that (i) at a given echo-path 

Table XIII — 1970 laboratory echo-test results — Approximately 
100 ratings (subjects) per condition 

Connection loudness loss = 18 dB 
Side tone-path loudness loss = 12 dB 

Circuit noise = 33 dBrnC 



Echo Path 


Percent of Subjects' Votes 


in Each Category 


Delay 

(ms) 


Loudness Loss 

(dB) 


Excel. 


Good 


Fair 


Poor 


Unsat. 


— 


33.0 


9.0 


49.0 


34.0 


8.0 


0.0 


600 


33.0 


1.0 


1.0 


2.0 


17.6 


78.4 




43.0 


2.0 


10.8 


19.6 


39.2 


28.4 




53.0 


3.9 


36.3 


44.1 


11.8 


3.9 




63.0 


6.9 


47.1 


28.4 


16.6 


1.0 




73.0 


6.0 


44.0 


42.0 


6.0 


2.0 


1200 


43.0 


1.0 


5.0 


19.0 


38.0 


37.0 




53.0 


6.0 


29.0 


40.0 


19.0 


6.0 




63.0 


4.0 


45.0 


36.0 


14.0 


1.0 




73.0 


7.0 


39.0 


39.0 


13.0 


2.0 
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delay, transmission quality improves with increasing echo-path loud- 
ness loss, and (ii) at a given echo-path loudness loss, transmission 
quality is degraded with increasing echo-path delay. 

4.4 1970 SIBYL echo test 

An echo test was conducted in early 1970 using sibyl. The purpose 
of the test was to enable comparison of results obtained from subjects 
conversing on actual telephone calls to results obtained under labora- 
tory conditions. The results of tests reported in Section 4.2 were used 
to guide selection of the conditions for the 1970 sibyl echo test. 

Forty-five subjects participated in these tests. Procedures followed 
by the subjects were the same as those for the loss-noise tests reported 
in Section 3.3. 

Test variables were echo-path delay (three values) and echo-path 
loudness loss (five values for each delay). In addition, a condition 
without echo was included as a reference. 

Connection loudness loss was 10 dB, the base condition for the 
Holmdel sibyl tests reported in Sections 3.2 and 3.3. Circuit noise 
was 30 dBrnC. Side tone path loudness loss was about 12 dB and 
average room noise was estimated to be about 42 dB(A). 

The test conditions and results are given in Table XIV. As with 
results of echo tests discussed in preceding sections, these results show 
that transmission quality is strongly dependent on echo-path delay 
and echo-path loudness loss. 

V. ANALYSIS OF INDIVIDUAL TEST DATA 

The raw test results from any individual test provide subjective- 
opinion information expressible in the form of percent of ratings in 
each of the five rating categories for each test condition. The utilization 
of these raw test results in this form for transmission planning is 
difficult because it is usually necessary to have ratings available for 
transmission parameter values not specifically included in the tests. 
Thus, some form of data analysis is frequently applied to obtain 
graphical or analytical representations of the data that are more 
convenient for use in transmission planning studies. This can involve 
simple curve fitting to the raw test data 5 - 6 or more elaborate models of 
subject ratings using binomial or other distributions. 20 

For example, Lewinski in Ref. 5 provides separate "smooth" fits 
for the percentage of responses in the categories excellent, good or 
better, fair or better, and poor or better as a function of circuit noise. 
Similarly Sen in Ref. 6 provides mathematical expressions and contours 
for the percent good or better and percent poor or worse as a function 
of connection loss and noise. A different approach is suggested by 
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Table XIV — 1970 sibyl echo test 

Connection loudness loss = 10 dB 

Circuit Noise = 30 dBrnC 



Echo Path 


Number 

of 
Ratings 


Percent of Subjects' Votes in Each Category 


Delay 
(ms) 


Loudness 
Loss 
(dB) 


Excel. 


Good 


Fair 


Poor 


Unsat. 


10.0 
36.0 
72.0 


CO 

5.8 
10.8 
15.8 
20.8 
25.8 

14.4 
19.4 
24.4 
29.4 
34.4 

22.4 
27.4 
32.4 
37.4 
42.4 


183 

51 
23 
38 
45 
41 

38 
37 
51 
31 
46 

35 
20 
34 
33 
46 


32.2 




15.8 

8.9 
19.5 



2.7 
11.8 

6.4 
19.6 


10.0 

5.9 
12.1 
13.1 


36.6 

15.7 
8.7 
21.0 
53.3 
51.2 

2.6 


35.3 
35.6 
36.9 

2.9 



8.8 
24.2 
47.8 


24.6 

29.4 
30.4 
39.5 
20.0 
29.3 

18.4 
29.7 
23.5 
25.8 
28.3 

2.9 
15.0 
29.4 
30.4 
30.4 


5.5 

47.1 
47.8 
21.0 
13.3 


50.0 
37.9 
23.5 
25.8 
13.0 

65.7 
60.0 
38.2 
24.2 
6.5 


1.1 

7.8 
13.1 
2.7 
4.5 


29.0 

29.7 

5.9 

6.4 

2.2 

28.5 

15.0 

17.7 

9.1 

2.2 



Prosser, Allnatt, and Lewis in Ref. 20 where they point out that five 
separate mathematical functions can be specified, one for each grade 
on the rating scale, but advocate the desirability of a more convenient 
and compact representation by means of a single mathematical model 
that embraces all five functions. They examined various models based 
on the binomial distributions as well as logistic and gaussian curves. 
They adopted the second-order binomial as the simplest adequate 
model to describe the opinion distribution found in their experiment. 
We also recognized the advantages of a single mathematical model 
to represent the distribution of opinion in the five rating categories. 
The normal density curve was selected as a basis for the model 
described in the following sections because it provided somewhat 
greater flexibility in accommodating a variety of standard deviations. 
Because of the availability of digital computers for the data analysis, 
the additional computational complexity associated with the normal 
distribution was not judged to be a problem. 

5.1 Analysis method 

The subjective-test results for each test condition of connection 
loudness loss, L e , and circuit noise, N, form a vote histogram containing 
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the proportion, P„ of ratings for each of the rating categories, i = 1, 
2, 3, 4, 5. Rating category 1 represents the unsatisfactory category, 2 
represents poor, 3 represents fair, 4 represents good, and 5 represents 
excellent. 

The Pi's for each test condition sum to unity. 

The values of P, for each test condition were used to calculate the 
mean opinion score (mos) and sample standard deviation (sigmos) as 
in eqs. (4) and (5), respectively. 



mos = £ iPi 

i-i 



SIGMOS 



-[£ 



?Pi - (mos) 2 



(4) 



(5) 



The vote histogram was represented by a normal density curve with 
mean, n, and standard deviation, a. The area under this curve was 
divided into five regions, each with area P\. The areas were defined as 
follows: from minus infinity to 1.5 as P\, from 1.5 to 2.5 as P\>, from 
2.5 to 3.5 as P 3 , from 3.5 to 4.5 as P i} and from 4.5 to infinity as P 5 ; 
the Pi's sum to unity. This quantization of the area under the normal 
curve into five discrete regions was the basis for using the normal curve 



100 




2 3 4 

MEAN. /x. OF A NORMAL-DENSITY CURVE 



Fig. 12 — MH loss-noise test data percentages as a function of the Step 3 normal- 
density means compared with percentages predicted using the MH standard devia- 
tion, 1.44. 
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100 



DATA 
A U 
O U + P 
V U+P+F 
D U+P+F+G 




1 2 3 4 5 6 

MEAN, fl. OF A NORMAL-DENSITY CURVE 

Fig. 13 — HOI loss-noise test data percentages as a function of the Step 3 normal- 
density means compared with percentages predicted using the HOl standard devia- 
tion, 1.02. 

to represent this type of data. These Pi's were used to compute the 
pseudo mean opinion scores (mosq) and sample standard deviations 
(sigmosq) in terms of the mean, /x, and standard deviation, a, of the 
normal curve as given in eqs. (6) and (7), respectively. 



.5-*»)/ff 



MOSq 



SIGMOSq 



H)4 



5 4 T 1 rU+0. 

i = i y-i L V2ir y— 

= I" t *A - (mosq) 2 ? 



(6) 



— (MOSq) 2 



(7) 



Step 1 in the four-step analysis was to find a normal density curve 
for each test condition such that mos = mosq and sigmos = sigmosq. 
These two constraints were used in an iterative computer procedure 
to determine the values of the mean, n, and standard deviation, a, of 
the normal density curve used to represent the data for each test 
condition. 
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Three other criteria were considered in the determination of a 
normal density curve during the development of this analysis pro- 
cedure. One criterion was to determine n and a such that the propor- 
tions good or better (P 4 + P5) and poor or worse (Pi + P 2 ) were the 
same for the data and the normal curve. Another was to determine n 
and a to minimize the sum of the squares of the differences between 
P, and P lt Pi + P 2 and A + P 2 , Pi + P 2 + Pa and P x + P 2 + P 3 , 
and finally Pi + P 2 + P 3 + P 4 and P\ + P 2 + P 3 + P 4 . The third 
criterion was to determine y. and a to minimize the sum of the square 
of the differences P, — Pi, i = 1,2, • • • , 5. The selection of mos = mosq 
and sigmos = sigmosq was chosen as the general criterion of fit, since 
the other criteria were found to be more sensitive to the experimental 
variability inherent in subjective data of this type, particularly when 
the number of subjects used is small (less than about 100). This 
selection was made after applying the several analysis criteria to several 
hundred sets of test data generated by Monte Carlo simulation. 

After all the test conditions were represented by a normal density 
curve with mean, /x, and standard deviation, <r, an average of the stan- 
dard deviations for all of the test conditions was computed in Step 2. 
In determining this average, the individual <r's were weighted in 
accordance with the number of votes per condition and by the weight- 



100 




12 3 4 5 6 

MEAN, fx, OF A NORMAL-DENSITY CURVE 

Fig. 14 — H02 loss-noise test percentage as a function of the Step 3 normal-density 
means compared with percentages predicted using H02 standard deviation, 0.998. 
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MEAN, fj., OF A NORMAL-DENSITY CURVE 

Fig. 15 — 1968 laboratory echo data percentages (noise=18 dBrnC) as a function 
of the Step 3 normal-density means compared with percentages predicted using the 
echo standard deviation, 0.888. 

ing given in eq. (8). This latter weighting was based on the analyses 
of the Monte Carlo data mentioned previously by examining the 
variations in the standard deviations as a function of the fit mean, /x. 



Weighting = 



1 + 



(m - 3) 5 



(8) 



The weighted average standard deviation, a a , was then used in 
Step 3 as the standard deviation for all of the normal density curves. 
Using a a , a new mean, n a , was computed for each test condition subject 
to the constraint, mos = mosq. The end result was a family of normal 
density curves, all with the same standard deviation and with means 
as determined above. 

Figures 12 to 19 illustrate the results from the first three steps in 
the procedure for several of the tests described in this paper. In these 
figures the cumulative percent of ratings in four categories — unsatis- 
factory, unsatisfactory plus poor, unsatisfactory plus poor plus fair, 
unsatisfactory plus poor plus fair plus good — are plotted against the 
fit mean, n„, determined in Step 3. The solid curves are plotted using 
the weighted average standard deviation, a a , from Step 2. Also shown 
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are the raw data plotted against the respective fit mean, n a , for each 
condition. These figures show that the normal density curves defined 
by the values, n a , and an average standard deviation, a a , provide 
a convenient and simple representation of the raw data for any 
single test. 

In Step 4, the means of all the normal curves are fitted by a suitable 
analytical function of the test parameters using a least-squares-fit 
technique. 

In summary, the steps involved in the analysis procedure can be 
described as follows : 

Step 1. A normal density curve is used to represent the vote histo- 
gram for each test condition such that mos = mosq and 

SIGMOS = SIGMOSq. 

Step 2. The standard deviations of the normal curves of Step 1 for 
all test conditions under consideration are weighted and 
averaged to obtain a single value for the standard devi- 
ation, a a . 

Step 3. The single value of standard deviation from Step 2 is used 
for each test condition as the standard deviation of the 




2 3 4 

MEAN. IX. OF A NORMAL-DENSITY CURVE 



Fig. 16 — 1968 laboratory echo data percentages (noise=28 dBrnC) as a function 
of the Step 3 normal-density means compared with percentages predicted using the 
echo standard deviation, 0.888. 
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2 3 4 

MEAN, fx, OF A NORMAL-DENSITY CURVE 



Fig. 17 — 1968 laboratory echo data percentages (noise=38 dBrnC) as a function 
of the Step 3 normal-density means compared with percentages predicted using the 
echo standard deviation, 0.888. 




2 3 4 

MEAN, fJL, OF A NORMAL-DENSITY CURVE 

Fig. 18 — sibyl echo data percentages as a function of the Step 3 normal-density 
means compared with percentages predicted using the echo standard deviation, 0.888. 
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A U 
O U + P 
V U+P+F 
D U+P+F+G 




2 3 4 5 6 

MEAN, fj., OF A NORMAL-DENSITY CURVE 

Fig. 19 — 1970 laboratory echo data percentages as a function of the Step 3 normal- 
density means compared with percentages predicted using the echo standard devia- 
tion, 0.888. 

corresponding normal curve, and the mean of the normal 
curve, n a , is adjusted such that mos = mosq. 
Step 4. The means of all normal curves of Step 3 are fitted, on a 
least-squares-error basis, to an appropriate function of the 
test parameters. 

The results at each step of this analysis are summarized in Table 
XV for H02 loss/noise data. 

In this table, the results presented for Step 4 are based on the 
analytical function [eq. (11) in the next section] fitted in Step 4. 
Comparison of the entries for Step 4 with the entries for the previous 
steps illustrate the extent to which this analytical function provides 
a good fit to the test results. For this test, all of the fit means in Step 4 
are within approximately 0.2 of the fit means in Step 3. The individual 
differences have a mean of 0.035 and a standard deviation of 0.13. 
This agreement is considered reasonable in view of the average stan- 
dard deviation of 0.998 and about 50 to 70 ratings per condition. Some- 
what larger differences were obtained in the MH and HOI tests where 
the number of ratings per test condition was smaller. 

5.2 Loss-noise analysis 

The results obtained by applying the analysis method to the data 
from the three loss/noise tests are given in eqs. (9), (10), and (11), 
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Table XV — Summary of results for H02 loss-noise test 
at each step in the analysis procedure 



Step in the 
Fit Process 



Percentage of Votes 



Excel. Good Fair Poor Unsat. 



Mean 
Opinion 
Score* 



Standard 
Deviation* 



Fit 
Mean 



Fit 
Sigma 



Test Condition 1, Loss = 5 dB, Noise = 25 dBrnC 



Raw Data 
Step 1 
Step 2 
Step 3 
Step 4 



49.49 


35.35 


13.13 


0.00 


2.02 


4.303 


51.54 


30.94 


14.15 


3.05 


0.32 


4.303 


51.71 


33.48 


12.77 


1.92 


0.11 


4.348 


49.16 


34.51 


13.96 


2.23 


0.14 


4.303 


50.00 


34.18 


13.56 


2.12 


0.13 


4.318 



0.846 
0.846 
0.782 
0.801 
0.795 



4.543 
4.543 
4.479 
4.500 



1.117 
0.998 
0.998 
0.998 



Test Condition 2, Loss = 10 dB, Noise = 25 dBrnC 



Raw Data 
Step 1 
Step 2 
Step 3 
Step 4 



47.23 


40.62 


10.20 


1.26 


0.68 


4.325 


48.79 


36.59 


12.98 


1.57 


0.06 


4.325 


48.88 


34.61 


14.10 


2.26 


0.15 


4.298 


50.40 


34.02 


13.38 


2.07 


0.13 


4.325 


50.00 


34.18 


13.56 


2.12 


0.13 


4.318 



0.762 
0.762 
0.803 
0.792 
0.795 



4.472 
4.472 
4.510 
4.500 



0.923 
0.998 
0.998 
0.998 



Test Condition 3, Loss = 20 dB, Noise = 25 dBrnC 





14.00 


26.00 


40.00 


18.00 


2.00 


3.320 


0.989 


_ 





Step 1 


11.81 


31.46 


36.85 


16.75 


3.14 


3.320 


0.989 


3.333 


0.985 


Step 2 


12.11 


31.24 


36.45 


16.88 


3.31 


3.320 


0.998 


3.333 


0.998 


Step 3 


12.11 


31.24 


36.45 


16.88 


3.31 


3.320 


0.998 


3.333 


0.998 


Step 4 


12.26 


31.38 


36.37 


16.73 


3.26 


3.326 


0.998 


3.340 


0.998 



Test Condition 4, Loss = 30 dB, Noise = 25 dBrnC 





3.23 


3.23 


32.26 


35.48 


25.81 


2.226 


0.974 


_ 





Step 1 


1.26 


8.78 


27.39 


36.40 


26.16 


2.226 


0.974 


2.166 


1.043 


Step 2 


0.97 


8.10 


27.83 


37.88 


25.23 


2.217 


0.946 


2.166 


0.998 


Step 3 


0.99 


8.24 


28.04 


37.82 


24.91 


2.226 


0.947 


2.176 


0.998 


Step 4 


1.03 


8.43 


28.34 


37.73 


24.47 


2.238 


0.949 


2.190 


0.998 



Test condition 5, Loss = 5 dB, Noise = 32 dBrnC 





27.14 


35.71 


31.43 


5.71 


0.00 


3.843 


0.889 








Step 1 


25.24 


40.95 


27.16 


6.17 


0.48 


3.843 


0.889 


3.885 




Step 2 


26.89 


38.13 


26.72 


7.42 


0.84 


3.828 


0.938 


3.885 


0.998 


Step 3 


27.45 


38.19 


26.35 


7.20 


0.80 


3.843 


0.935 


3.902 




Step 4 


25.42 


37.91 


27.70 


8.02 


0.95 


3.788 


0.945 


3.840 


0.998 



Test condition 6, Loss = 10 dB, Noise = 32 dBrnC 





18.97 


39.66 


32.76 


6.90 


1.72 


3.672 


0.917 










19.46 


39.06 


31.65 


8.93 


0.89 


3.672 


0.917 


3.700 


0.929 


Step 2 


21.14 


36.80 


30.60 


10.09 


1.37 


3.662 


0.965 


3.700 


0.998 


Step 3 


21.46 


36.91 


30.38 


9.91 


1.34 


3.672 


0.963 


3.711 


0.998 


Step 4 


27.72 


38.22 


26.17 


7.10 


0.79 


3.850 


0.933 


3.910 


0.998 



Test condition 7, Loss = 20 dB, Noise = 32 dBrnC 



Raw Data 


4.69 


6.25 


48.44 


34.38 


6.25 


2.688 


0.864 










1.40 


14.76 


42.65 


33.60 


7.59 


2.688 


0.864 


2.684 


0.826 


Step 2 


3.44 


17.24 


36.64 


30.91 


11.77 


2.697 


0.999 


2.684 


0.998 


Step 3 


3.37 


17.03 


36.53 


31.11 


11.97 


2.687 


0.999 


2.674 


0.998 


Step 4 


5.44 


21.94 


38.19 


26.39 


8.03 


2.904 


1.007 


2.900 


0.998 
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Table XV — Continued 



Step in the 
Fit Process 



Percentage of Votes 



Excel. Good Fair Poor Unsat 



Mean 
Opinion 
Score* 



Standard 
Deviation* 



Fit 
Mean 



Fit 

Sigma 



Test condition 8, Loss = 30 dB, Noise = 32 dBrnC 



Raw Data 


1.92 


7.69 


15.38 


51.92 


23.08 


2.135 


0.920 






Step 1 


0.67 


6.66 


25.95 


38.87 


27.85 


2.135 


0.920 


2.076 


0.981 


Step 2 


0.76 


6.92 


25.87 


38.26 


28.19 


2.138 


0.931 


2.076 


0.998 


Step 3 


0.75 


6.87 


25.78 


38.27 


28.33 


2.135 


0.930 


2.072 


0.998 


Step 4 


0.45 


4.89 


21.72 


38.15 


34.80 


1.980 


0.895 


1.890 


0.998 



Test condition 9, Loss = 5 dB, Noise = 42 dBrnC 



Raw Data 


11.32 


11.32 


35.85 


33.96 


7.55 


2.849 


1.088 






Step 1 


6.83 


20.85 


34.25 


26.55 


11.51 


2.849 


1.088 


2.839 


1.116 


Step 2 


4.80 


20.59 


37.91 


27.72 


8.98 


2.845 


1.005 


2.839 


0.998 


Step 3 


4.84 


20.67 


37.93 


27.63 


8.92 


2.849 


1.005 


2.843 


0.998 


Step 4 


5.23 


21.50 


38.11 


26.83 


8.34 


2.884 


1.006 


2.880 


0.998 



Test condition 10, Loss = 10 dB, Noise = 42 dBrnC 



Raw Data 


17.53 


17.53 


32.99 


24.74 


7.22 


3.134 


1.181 






Step 1 


14.29 


24.81 


30.53 


20.76 


9.60 


3.134 


1.181 


3.150 


1.265 


Step 2 


8.81 


27.48 


37.97 


20.83 


4.91 


3.144 


1.005 


3.150 


0.998 


Step 3 


8.63 


27.24 


38.02 


21.07 


5.03 


3.134 


1.006 


3.139 


0.998 


Step 4 


7.45 


25.51 


38.30 


22.83 


5.90 


3.058 


1.007 


3.060 


0.998 



Test condition 11, Loss = 20 dB, Noise = 42 dBrnC 



Raw Data 


4.35 


10.87 


19.57 


41.30 


23.91 


2.304 


1.081 






Step 1 


2.92 


11.45 


26.35 


31.67 


27.61 


2.304 


1.081 


2.217 


1.206 


Step 2 


1.11 


S.82 


28.91 


37.54 


23.62 


2.262 


0.953 


2.217 


0.998 


Step 3 


1.25 


9.51 


29.86 


37.16 


22.23 


2.304 


0.960 


2.263 


0.998 


Step 4 


1.24 


9.46 


29.79 


37.18 


22.32 


2.301 


0.959 


2.260 


0.998 



Test condition 12, Loss = 30 dB, Noise = 42 dBrnC 



Raw Data 


0.00 


0.00 


25.00 


0.00 


75.00 


1.500 


0.866 






Step 1 


0.98 


3.30 


9.20 


17.76 


68.75 


1.500 


0.866 


0.705 


1.626 


Step 2 


0.01 


0.25 


3.35 


17.68 


78.71 


1.252 


0.521 


0.705 


0.998 


Step 3 


0.05 


1.06 


8.84 


28.93 


61.12 


1.500 


0.705 


1.218 


0.998 


Step 4 


0.12 


1.98 


13.00 


33.70 


51.20 


1.661 


0.786 


1.470 


0.998 



* Mean opinion score and standard deviation are calculated from the percentage of votes given in 
each table for the corresponding step in the fit process. 

respectively, for the MH, HOI, and H02 tests. 

mmh = 11.54 - 0.1099 1 L e - 11.7| - O.I68JV1 - 0.001059LJV, 

a = 1.44, (9) 

mhoi = 7 - 0.1365 1 L e - 10.31| - 0.1219iV 2 + 0.001577L e iV2 

a = 1.02, (10) 

MH02 = 7.17 - 0.1681 \L e - 6.7 1 - 0.1058.V 3 + 0.002106LJV 3 

a = 0.998, (11) 
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where 

L e = Acoustic-to-acoustic loudness loss (in dB) of an overall tele- 
phone connection, determined using the Electro-Acoustic 
Rating System (ears) method. 
N = Circuit noise (in dBrnC) at the input to a set with a receive- 
loudness rating of 26 dB, determined using the ears method. 

Ni = Total noise in dBrnC resulting from power addition of the 
circuit noise, N, from the MH tests with 34.03 dBrnC. 

N2 = Total noise in dBrnC resulting from power addition of the 
circuit noise, N, from the HOI tests with 23.76 dBrnC. 

N 3 = Noise, N from the H02 tests. 

The values 34.03 and 23.76 were determined as fit parameters. The 
particular functional form was selected to provide as simple a model 
as possible of the systematic effects observed in the data. 

The results represented by eqs. (9), (10), and (11) revealed two 
important differences between the MH and HO tests. First, the 
standard deviation, <r, was considerably larger for the MH tests than 
for either HOI or H02. Second, the subjective opinions, as represented 
by the means, n, calculated from eqs. (9), (10), and (11) were con- 
siderably higher in the MH test compared with the HO tests. These 
differences occurred despite the similarities of the tests. A careful 
examination of either the raw data or the smooth results clearly shows 
that the subjects' ratings tended to be more critical in the two HO 
tests compared with the subjects in the MH test. 

A clearer picture of the differences is obtained by selecting a set of 
loss (L e ) and circuit noise (iV) values over a common range of the 
tests for MH, HOI, and H02 and computing the corresponding values 
of the means (n) for the three tests from eqs. (9), (10), and (11). If 
plots are made of the MH means versus both the HOI and H02 
means and the appropriate linear regression made for both plots, then 
eqs. (12) and (13), respectively, represent the regression line between 
the MH and HOI means and the MH and H02 means. Such a plot 
is shown in Fig. 20 for MH and HOI. 

jumh = 1.372/xhoi - 0.206. (12) 

mmh = 1.288mho2 - 0.215. (13) 

The Pearson product moment coefficient of correlation was found to 
be 0.9586 and 0.9693 for MH with HOI and H02, respectively. Eqs. 
(12) and (13) clearly show the difference between the means for the 
MH and the two HO tests. These equations also show the close 
agreement between the two Holmdel tests. 
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Fig. 20 — Comparison of 1965 Murray Hill and 1969 Holmdel 1 loss-noise results. 

Equations (12) and (13) are used to adjust for the difference be- 
tween the HO results and the MH results in Section VI of this paper. 

5.3 Echo analysis 

The four echo subjective tests yielded data on the subjective effects 
of echo-path loudness loss, E, in dB and echo-path delay, D, in ms. 
Circuit noise, A', was not the same for all four tests, and needed to be 
considered as a test variable. Loudness loss, L e , was a factor in only 
two of these tests (the other two were listening only tests) and, as a 
first approximation, it was decided to ignore L„ and concentrate on E, 
D, and N in the analysis. 

Preliminary analyses of the individual test data indicated that there 
were only relatively small differences in the absolute ratings among the 
1968-1970 tests. Thus, it was feasible to combine these data and use 
the analysis method described previously. The resulting equation 
relating the normal density means to the test variables, as realized 
at Step 4 in the analysis, was a function of E, D, and N, where N for 
this preliminary analysis was the actual noise at the telephone set 
terminals. The effect of noise was asymptotic. That is, the fit mean, n, 
was determined largely by the values of E and D when echo was the 
predominant impairment, while, for any value of D, increasing the 
value of E gradually led to the mean being solely determined by the 
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value of N. This asymptotic effect of the noise made it relatively simple 
to separate the effects of noise and echo in the function depicting the 
mean as a function of the test variables. The resulting functions are 
given in eq. (14). 

M *i = 4.74 - 3.32 log 10 [(1 + Z))/Vl + (D/480) 2 ] + 0.1414£ 
fi N = 6.38 - 0.094iV 

m = ^^ - V(^i LM ) 2 + (0 ^ (14) 

o- = 0.888. 

Although these functions were a useful interim result, subsequent 
analysis described in the later sections of this paper indicated that 
when the effects of both loss and noise were included and the noise was 
referred to the input of a reference set, further modifications were 
necessary. These functions are included here because they provided a 
basis for the subsequent modifications. 

VI. COMBINATION OF MODELS 

The results of the three loss-noise tests as given by eqs. (9), (10), 
and (11) showed fundamental differences among the tests. Despite 
the similarity of the tests and the general character of the results, 
both the raw data and the smoothed results showed that the subjects' 
ratings in the two HO tests tended to be more critical in their evalu- 
ations than they were in the MH tests. This could have occurred 
because of one or more differences in the tests, such as room noise, 
sidetone path loss, year of test, or some fundamental difference in the 
attitude of the subject groups. The frequent repetition of high-quality 
conditions in the Holmdel tests may also have been a factor. 

The exact reasons for the differences in the test results could not 
be determined. Because of these differences, direct pooling of results 
from the three tests did not appear to be justifiable. However, the 
test results were combined by adjusting the HOI and H02 results to 
a MH base using the linear transformation obtained from the linear 
regressions introduced previously in eqs. (12) and (13). In this way, 
the systematic differences among the test results were preserved, while 
achieving the advantages of a larger data base. The transformed means 
were then included with the MH means, and a new equation was 
obtained by applying Step 4 to this combined set of means. Table XVI 
shows the fit means at Step 3 for the MH, HOI, and H02 test results 
and the adjusted fit means for the HOI and the H02 test results. Thus, 
the final fit was based on a total of 51 conditions. The final result 
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is eq. (15). 

mmh = 10.36 - 0.185V(L e - 7.2) 2 + 1 - 0.1U7N F 

+ 0.00167LsJV F , (15) 

where 

N F = Power addition of noise with 27.37 dBrnC. 

6.7 General rating scale 

Equation (15) above is calculated in terms of the MH base. This 
equation together with eqs. (12) and (13) can be used to express the 
representation of the subjective ratings in terms of any one of the 
other test bases, HOI or H02. Each test base has a standard deviation 
associated with it that can then be used in conjunction with the 
computed means to calculate predicted vote histograms from the 
normal density curves. However, if this is done, the fit means and vote 
histograms will be different for each test in accordance with the 
difference in absolute ratings obtained for each test. To eliminate the 
need for three separate equations, one for each test, a general trans- 
mission-rating scale was established. 

This transmission rating scale, referred to as the 22-scale, is simply 
a linear transformation of the normal density means, defined by eq. 
(15), with the constraints that two preselected transmission conditions 
are to be the anchor conditions for the transformation. 72-scale values 
of 80 and 40, respectively, were selected for the transmission conditions 
L„ = 15, N = 25, and L e = 30, N = 40. These two transmission con- 
ditions were selected to be well separated in quality. The first pair is 
typical of a short intertoll connection and the latter represents an 
extreme condition of loss and noise that should rarely occur even on 
long intertoll connections between long loops. 

Using the above transmission conditions as anchors, .R-scale values 
can be specified in terms of n for each test through the linear trans- 
formation R = a + bfi, with a and b determined from the anchor 
constraints. 

6.2 Loss-noise model 

From eq. (15), the transmission condition L E = 15, N = 25 yields 
/imh = 4.806 and L e = 30, N = 40 yields mmh = 1.528. Using these 
values of /x, respectively, with .R-scale values of 80 and 40 determines 
the transformation to the .ft-scale from the n scale as given in eq. (16). 

R = 21.37 + 12.20/xmh. (16) 

Substituting eq. (15) for /j M h into eq. (16) gives eq. (17), which is 
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CONNECTION LOUDNESS LOSS IN dB 



Fig. 21 — MH loss-noise test, Step 3 means from data as a function of loudness loss 
and noise compared with means predicted from the loss-noise model at the MH base. 

the .R-scale representation of the subjective opinion for loss and noise. 

R LN = 147.76 - 2.257V(L e - 7.2) 2 + 1 - 2.009JV, + 0.020371,^. 

(17) 

Equation (17) is plotted in Fig. 1 as transmission rating versus L e for 
a selected set of values of N . These curves represent the predicted 
transmission rating, in terms of the .R-scale, for selected values of 
L e and N. 

The i?-scale result of eq. (17) can also be used in conjunction with 
the appropriate standard deviations associated with eqs. (9), (10), 
and (11) and the appropriate inverse linear regression lines of eqs. 
(12) and (13) to calculate percent good or better, poor or worse, or 
other characteristics at the chosen test base. 

For the /z-scale, the proportion of ratings good or better is the 
integral of the standard normal density curve from (3.5 — n)/o to 
infinity. In the R-scale, this corresponds to the integral of the standard 
normal density curve from minus infinity to [R — (a + 3.56)]/<r&. 
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Similar computations can be made for proportion poor or worse, or 
for proportions of ratings in any of the five categories. The appropriate 
limits of integration to compute the proportion of ratings good or 
better and poor or worse are given in Table II for the three loss-noise 
test bases (MH, HOI, H02). 

The above discussion concerning the relationships between the 
proportions good or better and poor or worse and the #-scale lead to 
the plots of Fig. 4 which show these relationships for the three 
test bases. 

Finally, the results summarized in Table II were also used to 
generate curves showing the tradeoff between L e and N for selected 
values of percent good or better and percent poor or worse as shown 
in Figs. 6 and 7, respectively. As noted on the figures, these results 
correspond to the MH data base. 

In Figs. 21 to 23, the third-step fit means for the individual tests 
are plotted as a function of loudness loss with circuit noise as a param- 
eter. The solid lines in the figure correspond to the values of trans- 
mission rating in the final model transformed by the appropriate 
relation between n and R for each test. These figures show a generally 
good fit to the individual test results for each of the tests. 




10 15 20 

CONNECTION LOUDNESS LOSS IN dB 



Fig. 22 — HOI loss-noise test, Step 3 means from data as a function of loudness loss 
and noise compared with means predicted from the loss-noise model at the HOI base. 
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O 32dBrnC NOISE 
A 42dBrnC NOISE 
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30 



Fig. 23 — H02 loss-noise test, Step 3 means from data as a function of loudness loss 
and noise compared with means predicted from the loss-noise model at the H02 base. 

6.3 Talker-echo model 

The 72-scale was introduced previously in terms of L e and N. Thus, 
it was necessary to use the conversational echo test results which 
included both loss and noise as parameters to establish an expression 
for the echo results on the i?-scale. The 1970 sibyl test was used as a 
basis for this conversion. As indicated in Table XIV, this test included 
a base condition with L e = 10 dB and N = 30 dBrnC for which the 
fit mean, /x, was 4.01. The /?-scale value corresponding to this combi- 
nation of loss and noise is 83.47. Similarly, the results for the noise 
condition in the 1968 lab tests were taken into account but given less 
weight because they were obtained in a less realistic test environment. 
A further aid in deriving this conversion was the HOI sibyl test for 
loss and noise which preceded the 1970 sibyl echo test and was 
conducted in the same manner with the same subjects. The final 
conversion is given in eq. (18) below. 



R = 18.7 + 16.1#»ji. 



(18) 



This relationship is almost identical to that of the HOI sibyl tests 
at low values of p. However, a slight correction was included for higher 
values of m to take account of the base condition in the 1970 sibyl 
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echo test. For the HOI test the value of n corresponding to the base 
condition (R = 83.47) was 3.86. The relationship in eq. (18) gives a 
value of 4.02 and provides good agreement with the actual value of n 
which was 4.01 in the 1970 sibyl echo tests. 

Substituting eq. (14) for n E i into eq. (18) yields eq. (19) which is 
the transmission-rating model for echo. 



R B = 95.01 - 53.45 log 10 [(1 + D)/<\ + (D/480) 2 ] + 2.277.E. (19) 

Equation (19) is plotted in Fig. 2. 

In the analysis above, eq. (19) was derived from eqs. (14) and (18) 
to provide excellent agreement for the 1970 sibyl echo tests. However, 
these same relationships did not provide good agreement with the 1970 
lab tests which had a higher loss included in all conditions. The base 
condition for this test as given in Table XII, with L e = 18 dB and 
N = 33 dBrnC (R = 67.36), had a fit mean, n, of 3.59. 

The equation, 

R = 20 + 13.33/i B2 , (20) 

provided a good match at this point and retained the relationship 
between n and R at low values of n obtained previously for the sibyl 
echo tests. With the relationship defined by eq. (20), the transmission- 
rating model for echo given in eq. (19) provided an excellent repre- 
sentation of the results from the 1970 lab test for echo. The extent of 
the agreement is illustrated in Section 6.4 where the combined loss- 
noise-echo model is discussed. 

6.4 Loss-noSse-echo model 

In the development of the echo results, it was noted that the deg- 
radations considered were echo-path-loudness loss (E), echo-path 
delay (D), loudness loss (L e ), and noise (N). Loss and noise were 
eliminated in the final echo result because it was felt that for any 
combined result, the loss and noise influence should be based on the 
larger data base available from the sibyl tests. 

The original analysis of the echo-loss-noise data showed that the 
loss and noise really only affected the circuit performance as an 
asymptote. That is, for very large E, the ratings were determined by 
L e and N. Use was made of this fact when the echo result of eq. (14) 
was developed. The combination of Rln and Re was made such that 
this asymptotic behavior was retained in the final model. 

The final result for loss, noise, and echo is presented in terms of the 
iE-scale as Rlne- This final result is shown in eq. (21). 



Rlne = Rm + Rm _ ,j( **»-** )'+"(» (21) 
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Fig. 24—1968 laboratory echo test (noise=18 dBrnC, loss=10 dB) Step 3 means 
from data as a function of echo-path loss and delay compared with means predicted 
from the loss-noise-echo model at the echo 1 base. 
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Fig. 25—1968 laboratory echo test (noise=28 dBrnC, loss=10 dB) Step 3 means 
from data as a function of echo-path loss and delay compared with means predicted 
from the loss-noise-echo model at the echo 1 base. 
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Fig. 26—1968 laboratory echo test (noise=38 dBrnC, loss=10 dB) Step 3 means 
from data as a function of echo-path loss and delay compared with means predicted 
from the loss-noise-echo model at the echo 1 base. 
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Fig. 27 — sibyl echo test, Step 3 means from data as a function of echo-path loss 
and delay compared with means predicted from the loss-noise-echo model at the 
echo 1 base. 
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ECHO-PATH LOUDNESS LOSS. E, IN dB 

Fig. 28 — 1970 laboratory echo test, Step 3 means from data as a function of echo 
path loss and delay compared with means predicted from the loss-noise-echo model 
at the echo 2 base. 



With C = 0, Rlne is simply the lesser of Rln and Re. The factor C 
is selected to represent the additional degradation when Rlk and Re 
are nearly equal. The value of C = 10 was based on echo tests that 
included echo, loss, and noise and was obtained as the product of the 
constant, 0.627, in eq. (14) and the slope of the line relating R and m 
given in eq. (18). 

The final result in terms of the 22-scale is : 



B m = Rln + Re - ^ L \ "" j + (10)2. (22) 

For high echo-path-loudness loss, the ratio is determined mainly by 
connection loudness loss and circuit noise and the result reduces to 
the Rln result. Similarly, for connection loudness loss near optimum 
and low circuit noise, the rating is determined mainly by the echo, and 
the result effectively reduces to the Re result. 

Comparison of the final model for loss, noise, and echo with the 
third-step-fit means for the individual tests are shown in Figs. 24 to 
28. As in the case of the loss-noise model, the final loss-noise-echo model 
provides good agreement with the test results from each of the in- 
dividual tests. 
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APPENDIX A 

Examples Demonstrating Use of the Transmission-Rating Models 
A.1 Example 1 — local connection 

500-type telephone sets on a calling and a called customer loop, each 
of which consists of 1 kilofoot of 26-gauge nonloaded cable : 

Loss (L e ) : tlr = -23.6 dB 
rlr = 26.3 dB 
L e = (tlr + rlr) dB 
= 2.7 dB 

Noise (N) : N T = Total noise at each telephone set from two 
loops, each of which meet the 20 dBrnC 
loop-noise objective. 5 
= 23 dBrnC 
N = Total noise referred to a telephone set with 
rlr of 26 dB 
= 27 dBrnC. 

Talker Echo (assumed negligible) : 
Using Table II with L e = 2.7 dB and N = 27 dBrnC, 

Rln = 78.3. 
Using Table II for the MH data base, 

GoB = 79.2%, PoW = 6.6%. 

A.2 Example 2 — local connection 

500-type telephone sets and a calling and a called customer loop, 
each of which consists of 8 kilofeet of 26-gauge nonloaded cable: 

Loss (L e ) : tlr = - 18.9 dB 
rlr = 27.3 dB 
L e = 8.4 dB 

Noise (N) : Assumed to be the same as for Example 1 : 
N = 27 dBrnC. 

1368 THE BELL SYSTEM TECHNICAL JOURNAL, NOVEMBER 1976 



Talker echo (assumed negligible) : 
Using Table II with L e = 8.4 dB and N = 27 dBrnC, 

Rur = 88.7. 
Using Table II for the MH data base, 

GoB = 92%, PoW = 1.8%. 

A.3 Example 3 — local connections 

500-type telephone sets on a calling and a called customer loop, 
each of which consists of 15 kilofeet of 26-gauge nonloaded cable: 

Loss (L e ) : tlr = - 13.2 dB 
rlr = 30.1 dB 
L e = 16.9 dB 

Noise (N) : Assumed to be the same as for Example 1. 

iV = 27 dBrnC. 

Talker echo (assumed negligible) : 

Using Table II with L e = 16.9 dB and N = 27 dBrnC, 

Rln = 75.5. 

Using Table II for the MH data base, 

GoB = 74.2%, PoW = 8.9%. 

A.4 Example 4 — toll connection 

500-type telephone sets on a calling and a called customer loop, each 
of which consists of 8 kilofeet of 26-gauge nonloaded cable : 

Loss(L e ): tlr = -18.9 dB 
rlr = 27.3 dB 

L = Class 5 office-to-Class 5 office loss 

= 7.7 dB (mean for the connection-length cate- 
gory, 775 to 2900 miles, Table VI of Ref. 12) 

L c = TLR + RLR + 7.7 

= 16.1 dB. 

Noise (A r ) : Assume no noise from the loops : 

Nt = Total noise from the Class 5 office-to-Class 5 
office connection (33.8 dBrnC for the connec- 
tion length category 775 to 2900 miles, Table 
III of Ref. 12 referred to the telephone set) 
= 28.5 dBrnC 
N = 32.5 dBrnC. 
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Talker echo (assumed negligible) : 
Using Table II with L e = 16.1 dB and N = 32.5 dBrnC, 

Rln — 71. 
Using Table II for the MH data base, 

GoB = 65.2%, PoW = 13.9%. 

A.5 Example 5 — toll connection 

Same as Example 4, Section A.5, except that it takes into account 
talker echo : 

Echo-path delay (D) 

D = Far-end echo-path loss 

= 37.3 ms (mean for connection length 1450 to 2900 miles, 
Table III of Ref. 11). 

Echo-path loudness loss (E) 

E' = Loss of echo path from near-end Class 5 office to distant 
end and return, 
= 23.3 dB (mean for connection length 1450 to 2900 miles, 
Table II of Ref. 11). 
E = tlr + rlr + E' 
= 31.7 dB. 

Using Table II with L a = 16.1 dB, N = 32.5 dBrnC, D - 37.3 ms, 
and E = 31.7 dB. 

Rlk = 71 
R E = 82.6 
Rlne = 65.2. 

Using Table II for the MH data base, 

GoB = 52.6%, PoW = 22.4%. 
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